Search for: All records

Regular expression (regex) coding has advantages for text analysis. Humans are often able to quickly construct intelligible coding rules with high precision. That is, researchers can identify words and word patterns that correctly classify examples of a particular concept. And, it is often easy to identify false positives and improve the regex classifier so that the positive items are accurately captured. However, ensuring that a regex list is complete is a bigger challenge, because the concepts to be identified in data are often sparsely distributed, which makes it difficult to identify examples of \textit{false negatives}. For this reason, regex-based classifiers suffer by having low recall. That is, it often misses items that should be classified as positive. In this paper, we provide a neural network solution to this problem by identifying a \textit{negative reversion set}, in which false negative items occur much more frequently than in the data set as a whole. Thus, the regex classifier can be more quickly improved by adding missing regexes based on the false negatives found from the negative reversion set. This study used an existing data set collected from a simulation-based learning environment for which researchers had previously defined six codes and developed classifiers with validated regex lists. We randomly constructed incomplete (partial) regex lists and used neural network models to identify negative reversion sets in which the frequency of false negatives increased from a range of 3\\%-8\\% in the full data set to a range of 12\\%-52\\% in the negative reversion set. Based on this finding, we propose an interactive coding mechanism in which human-developed regex classifiers provide input for training machine learning algorithms and machine learning algorithms ``smartly" select highly suspected false negative items for human to more quickly develop regex classifiers. 

Learning analytics uses large amounts of data about learner interactions in digital learning environments to understand and enhance learning. Although measurement is a central dimension of learning analytics, there has thus far been little research that examines links between learning analytics and assessment. This special issue of Computers in Human Behavior highlights 11 studies that explore how links between learning analytics and assessment can be strengthened. The contributions of these studies can be broadly grouped into three categories: analytics for assessment (learning analytic approaches as forms of assessment); analytics of assessment (applications of learning analytics to answer questions about assessment practices); and validity of measurement (conceptualization of and practical approaches to assuring validity in measurement in learning analytics). The findings of these studies highlight pressing scientific and practical challenges and opportunities in the connections between learning analytics and assessment that will require interdisciplinary teams to address: task design, analysis of learning progressions, trustworthiness, and fairness – to unlock the full potential of the links between learning analytics and assessment. 

The present conceptual literature review analyzes 50 studies that systematically examined the effects of authentic learning settings on cognitive or motivational learning outcomes. The analysis focuses on describing the context of the studies, the design elements of authentic learning settings, and the pursued intentions of authenticity. The review further describes the effects of authentically designed learning settings on cognitive outcomes, motivational outcomes, and learners’ perceived authenticity revealed by previous research. Building on these findings, we conducted Epistemic Network Analysis (ENA) of contrasting cases to identify design elements and intentions of authenticity characterizing studies that show high effectiveness for cognitive and motivational outcomes versus those with low effectiveness. The ENA results suggest, for instance, that providing authentic materials (as a design element of authentic learning settings) to resemble real-life experiences (as an intention of authenticity) could be a double-edged sword, as they feature both authentically designed learning settings with low effects on cognitive outcomes and settings with high effects on motivational outcomes. Overall, the results of the present literature review point to critical limitations of previous research, such as a lack of clear definitions and operationalizations of authentic learning. Consequently, we draw specific conclusions about how future research could improve our understanding of how to create and implement powerful methods of authentic learning. 

Regular expression (regex) based automated qualitative coding helps reduce researchers’ effort in manually coding text data, without sacrificing transparency of the coding process. However, researchers using regex based approaches struggle with low recall or high false negative rate during classifier development. Advanced natural language processing techniques, such as topic modeling, latent semantic analysis and neural network classification models help solve this problem in various ways. The latest advance in this direction is the discovery of the so called “negative reversion set (NRS)”, in which false negative items appear more frequently than in the negative set. This helps regex classifier developers more quickly identify missing items and thus improve classification recall. This paper simulates the use of NRS in real coding scenarios and compares the required manual coding items between NRS sampling and random sampling in the process of classifier refinement. The result using one data set with 50,818 items and six associated qualitative codes shows that, on average, using NRS sampling, the required manual coding size could be reduced by 50% to 63%, comparing with random sampling. 

In quantitative ethnography (QE) studies which often involve large da-tasets that cannot be entirely hand-coded by human raters, researchers have used supervised machine learning approaches to develop automated classi-fiers. However, QE researchers are rightly concerned with the amount of human coding that may be required to develop classifiers that achieve the high levels of accuracy that QE studies typically require. In this study, we compare a neural network, a powerful traditional supervised learning ap-proach, with nCoder, an active learning technique commonly used in QE studies, to determine which technique requires the least human coding to produce a sufficiently accurate classifier. To do this, we constructed multi-ple training sets from a large dataset used in prior QE studies and designed a Monte Carlo simulation to test the performance of the two techniques sys-tematically. Our results show that nCoder can achieve high predictive accu-racy with significantly less human-coded data than a neural network. 

This paper explores methodological questions in the study of identity through an examination and discussion of the empirical papers in this special issue. Particular attention is paid to the ways identity is operationalized in the study of how learning environments foster changes in students’ sense of self. The paper concludes that identity is a difficult construct to study in the context of learning environments because it is simultaneously performative and subjective, and these dual aspects of identity may be best operationalized in an interactional view, in which identity is conceptualized as a set of relations between aspects of identity rather than as a state that can be coded directly in data on learning. 

In this paper, we describe iPlan, a web-based software platform for constructing localized, reduced-form models of land-use impacts, enabling students, civic representatives, and others without specialized knowledge of land-use planning practices to explore and evaluate possible solutions to complex, multi-objective land-use problems in their own local contexts.